Three-Dimensional Shape Measuring Method And Three-Dimensional Shape Measuring Device

ABSTRACT

A three-dimensional shape measuring method includes: projecting a first grid pattern based on a first light and a second grid pattern based on a second light onto a target object in such a way that the first grid pattern and the second grid pattern intersect each other, the first light and the second light being lights of two colors included in three primary colors of light; picking up, by a three-color camera, an image of the first grid pattern and the second grid pattern projected on the target object, and acquiring a first picked-up image based on the first light and a second picked-up image based on the second light; and performing a phase analysis of a grid image with respect to at least one of the first picked-up image and the second picked-up image and calculating height information of the target object.

The present application is based on, and claims priority from JPApplication Serial Number 2020-151912, filed Sep. 10, 2020, thedisclosure of which is hereby incorporated by reference herein in itsentirety.

BACKGROUND 1. Technical Field

The present disclosure relates to a three-dimensional shape measuringmethod and a three-dimensional shape measuring device.

2. Related Art

As a method for measuring a three-dimensional shape of an object, a gridprojection method using an optical technique is known.

WO2016/001985 discloses a measuring method for measuring a position ofan object surface, based on a captured image acquired by a cameracapturing a grid image of the object surface. This measuring methodincludes: a step of inputting a captured image captured by the camera inthe state where one cycle of a grid in a grid image is made tocorrespond to N pixels; a step of extracting a plurality of successivepixels in the inputted captured image; a step of finding a phase of afrequency component having a cycle of N pixels from an image of theplurality of pixels that are extracted; and a step of finding a positionof an object surface based on the phase. N is an integer greater than 2.

In this measuring method, the phase of the grid image is analyzed basedon image data corresponding one cycle of the grid image and thethree-dimensional shape of the object surface is thus measured. Themethod of analyzing the phase based on image data corresponding to onecycle of the grid image is particularly referred to as a “one-pitchphase analysis method (OPPA method)”. The one-pitch phase analysismethod enables high-speed analysis of a phase distribution based on onecaptured image.

In the measuring method in which the phase of the grid image is analyzedbased on image data corresponding to one cycle of the grid image, asdescribed in WO2016/001985, the measurement resolution is anisotropic.Specifically, in a direction parallel to the direction of the grid, themeasurement resolution is equivalent to one pixel of the camera, whereasin a direction orthogonal to the direction of the grid image, themeasurement resolution is as low as equivalent to one cycle of the gridimage. Therefore, this measuring method has a problem in that themeasurement accuracy may not be sufficient, depending on the shape ofthe object.

SUMMARY

A three-dimensional shape measuring method according to an applicationexample of the present disclosure includes: projecting a first gridpattern based on a first light and a second grid pattern based on asecond light onto a target object in such a way that the first gridpattern and the second grid pattern intersect each other, the firstlight and the second light being lights of two colors included in threeprimary colors of light; picking up, by a three-color camera, an imageof the first grid pattern and the second grid pattern projected on thetarget object, and acquiring a first picked-up image based on the firstlight and a second picked-up image based on the second light; andperforming a phase analysis of a grid image with respect to at least oneof the first picked-up image and the second picked-up image andcalculating height information of the target object.

A three-dimensional shape measuring device according to anotherapplication example of the present disclosure includes: a projectorprojecting a first grid pattern based on a first light and a second gridpattern based on a second light onto a target object in such a way thatthe first grid pattern and the second grid pattern intersect each other,the first light and the second light being lights of two colors includedin three primary colors of light; a three-color camera picking up animage of the first grid pattern and the second grid pattern projected onthe target object and acquiring a first picked-up image based on thefirst light and a second picked-up image based on the second light; anda computing unit performing a phase analysis of a grid image withrespect to at least one of the first picked-up image and the secondpicked-up image and calculating height information of the target object.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of a three-dimensional shape measuring deviceaccording to a first embodiment.

FIG. 2 is a side view schematically showing an optical system of thethree-dimensional shape measuring device shown in FIG. 1.

FIG. 3 is a top view schematically showing the optical system of thethree-dimensional shape measuring device shown in FIG. 1.

FIG. 4 is a flowchart for explaining a three-dimensional shape measuringmethod according to the first embodiment.

FIG. 5 shows a picked-up image acquired by picking up, by a camera, animage of a grid pattern projected by a projector, and separating a firstlight component.

FIG. 6 shows a picked-up image acquired by picking up, by the camera, animage of the grid pattern projected by the projector, and separating asecond light component.

FIG. 7 is a schematic view showing a first grid pattern and a secondgrid pattern when the direction of arrangement of projector pixels isinclined to an x-axis and a y-axis.

FIG. 8 shows an example of a picked-up image acquired when only a gridpattern in one direction is projected onto four rod-like elements placedon a plane.

FIG. 9 shows an example of a picked-up image acquired when only a gridpattern in one direction is projected onto four rod-like elements placedon a plane.

FIG. 10 is a graph prepared by slicing out a part of a distribution ofheight information found from the picked-up image shown in FIG. 8.

FIG. 11 is a graph prepared by slicing out a part of a distribution ofheight information found from the picked-up image shown in FIG. 9.

FIG. 12 explains a procedure for finding the correlativity between aluminance value data set acquired from a one-pitch grid and a pluralityof sine waves generated with the phase shifted.

FIG. 13 is a table showing the absolute values of correlationcoefficients between a luminance value data set DS for each camera pixeland a plurality of sine waves, calculated from the table shown in FIG.12, and a maximum value thereof.

FIG. 14 is a flowchart for explaining a three-dimensional shapemeasuring method according to a second embodiment.

FIG. 15 shows a picked-up image acquired by picking up, by the camera,an image of a first grid pattern, a second grid pattern, and anall-pixel irradiation pattern projected by the projector, and separatinga third light component.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

The three-dimensional shape measuring method and the three-dimensionalshape measuring device according to the present disclosure will now bedescribed in detail, based on embodiments shown in the accompanyingdrawings.

1. First Embodiment

First, a three-dimensional shape measuring method and athree-dimensional shape measuring device according to a first embodimentwill be described.

1.1. Configuration of Device

FIG. 1 is a schematic view of the three-dimensional shape measuringdevice according to the first embodiment. In FIG. 1, an x-axis, ay-axis, and a z-axis are set as three axes orthogonal to each other.Each axis is represented by an arrow. The head side of the arrow isdefined as “positive side”. The base side is defined as “negative side”.In the description below, the negative side on the z-axis is referred toas “up” and the positive side on the z-axis is referred to as “down”. Aposition along the z-axis is referred to as “height”. A position withinan x-y plane is simply referred to as “position”.

A three-dimensional shape measuring device 1 shown in FIG. 1 is a devicethat calculates the position and the height of a surface of an object 9(target object) provided at a reference surface 91, that is, theposition and the height of an object surface 92, and measures thethree-dimensional shape thereof. A grid projection method is used tomeasure the three-dimensional shape. In the grid projection method, agrid pattern is projected onto the object 9 and an image of this stateis picked up. Then, a phase analysis is performed on the picked-upimage. By performing the phase analysis, the phase of the grid patternat each pixel can be found. Then, height information is calculated fromthe phase. The three-dimensional shape of the object surface 92 is thusfound.

The three-dimensional shape measuring device 1 shown in FIG. 1 has aprojector 2, a camera 3, and a control device 4. Also, a mounting table90 whose top surface is the reference surface 91, and the object 9placed at the reference surface 91, are illustrated in FIG. 1.

The projector 2 has a function of emitting at least lights of twocolors, of the three primary colors of light, red (R), green (G), andblue (B), and thus projecting a desired pattern. In this specification,the “color” refers to one of the three primary colors of light. In thisspecification, the three primary colors of light may be referred to as“RGB” according to need.

The projector 2 shown in FIG. 1 has a light source 21, a lightmodulation element 22 such as a liquid crystal display element, and alens 23. The projector 2 shown in FIG. 1 is a simplified version of theactual structure. For example, in practice, the projector 2 may differfrom the structure shown in FIG. 1 in that the light modulation element22 is separated into a plurality of light modulation elements.

In the projector 2, the light modulation element 22 spatially modulateslight emitted from the light source 21 and thus forms a grid pattern.This grid pattern is projected onto the object 9 via the lens 23. Theprojector 2 is electrically coupled to the control device 4. Thisenables the control device 4 to control the color, direction, pitch andthe like of the grid pattern projected from the projector 2. In thedescription below, a pixel of the light modulation element 22 is alsoreferred to as “projector pixel”.

The camera 3 is a three-color camera having a function of detecting theluminance of the three primary colors of light at each pixel andacquiring a two-dimensional distribution of the resulting luminancevalue.

The camera 3 shown in FIG. 1 has an image pickup element 31 and a lens32. The camera 3 shown in FIG. 1 is a simplified version of the actualstructure. For example, in practice, the camera 3 may differ from thestructure shown in FIG. 1 in that the image pickup element 31 isseparated into a plurality of image pickup elements.

In the camera 3, the image pickup element 31 picks up, via the lens 32,an image of the grid pattern projected on the object 9. The camera 3 iselectrically coupled to the control device 4. The picked-up image pickedup by the camera 3 is transmitted to the control device 4 and is usedfor phase analysis. In the description below, a pixel of the imagepickup element 31 is also referred to as “camera pixel”.

The control device 4 has a control unit 41, a computing unit 42, astorage unit 43, and a display unit 44.

The control unit 41 controls operations such as the projection of a gridpattern by the projector 2 and the image pickup of the grid pattern bythe camera 3 in such a way that these operations cooperate with eachother.

The computing unit 42 performs a phase analysis on the picked-up image.The computing unit 42 thus finds the phase of the grid pattern at eachcamera pixel and calculates the three-dimensional shape of the objectsurface 92.

The storage unit 43 stores control data of the grid pattern projected bythe projector 2, the picked-up image picked up by the camera 3, theresult of the computation by the computing unit 42, and the like.

The display unit 44 is provided according to need and displays thepicked-up image picked up by the camera 3, the result of the computationby the computing unit 42, and the like.

A part or the entirety of the control unit 41, the computing unit 42,and the storage unit 43 is formed of hardware having a processorprocessing information, a memory storing a program and data, and anexternal interface. The processor reads and executes various programsand data stored in the memory and thus implements each function.

The processor may be, for example, a CPU (central processing unit), aDSP (digital signal processor), or the like. The memory may be, forexample, a volatile memory such as a RAM (random-access memory), anon-volatile memory such as a ROM (read-only memory), a removableexternal storage device, or the like. The external interface may be, forexample, a wired LAN (local area network), a wireless LAN, or the like.

A part or the entirety of the control unit 41 and the computing unit 42may be implemented by hardware such as an LSI (large-scale integration),an ASIC (application-specific integrated circuit), or an FPGA(field-programmable gate array).

In the three-dimensional shape measuring device 1 as described above,two grid patterns in different directions from each other aresimultaneously projected onto the object 9, using lights of at least twocolors, as will be described in detail later. Then, an image of the gridpatterns projected on the object 9 is picked up and a phase analysis ofthe grid image is performed for each color. The three-dimensional shapeof the object surface 92 is found, using the result of the analysisacquired from the picked-up image of at least one color.

An example of the phase analysis method for the grid image is theone-pitch phase analysis method. The principle of this method isdescribed in WO2016/001985.

In the one-pitch phase analysis method, each of the optical system ofthe projector 2 and the optical system of the camera 3 is parallel tothe reference surface 91. Such an optical system is also referred to asMoiré topography optical system. In such an optical system, on apicked-up image acquired by the camera 3 picking up an image of a gridpattern projected on the object 9, the cycle of the grid pattern isconstant regardless of the height of the object surface 92. Meanwhile,the phase of the grid pattern at camera pixels changes depending on theheight of the object surface 92. Therefore, the three-dimensional shapeof the object surface 92 can be found by analyzing the coordinates ofeach camera pixel and the phase of the grid pattern at each camerapixel, using the one-pitch phase analysis method.

The phase analysis method like the one-pitch phase analysis method canperform a phase analysis based on a luminance distribution of one gridpitch in one picked-up image. Therefore, this phase analysis method isadvantageous in that it can find a phase distribution even when theobject 9 is moving. Meanwhile, in an analysis method that requires aplurality of picked-up images, for example, as in the phase shiftmethod, it is difficult to perform accurate three-dimensional shapemeasurement on the moving object 9.

FIG. 2 is a side view schematically showing the optical system of thethree-dimensional shape measuring device 1 shown in FIG. 1.

As shown in FIG. 2, in the three-dimensional shape measuring device 1, agrid pattern is projected in such a way as to spread from the center ofthe lens 23 of the projector 2. Here, the center of the lens 23 isdefined as a principal point O2 of the projector 2. Similarly, the imagepickup range of the camera 3 is a range spreading from the center of thelens 32. Here, the center of the lens 32 is defined as a principal pointO1 of the camera 3.

In FIG. 2, a grid pattern is schematically expressed by multiplestraight lines. Of the straight lines expressing the grid pattern, asolid line represents, for example, the optical path of light projectingan area where the luminance of the grid pattern is high, and a dashedline represents the optical path of light projecting an area where theluminance of the grid pattern is low.

As can be seen from FIG. 2, in the optical system of thethree-dimensional shape measuring device 1, one cycle of the gridpattern appears in the same size in the picked-up image by the camera 3no matter what height the reference surface 91 or the object surface 92is at. That is, the size of one cycle of the grid pattern in thepicked-up image is defined by internal parameters of the projector 2 andthe camera 3 and is not influenced by the distance to the referencesurface 91 or the object surface 92. Therefore, this optical systemenables the three-dimensional shape measurement on the object 9regardless of the distance to the reference surface 91 or the objectsurface 92.

FIG. 3 is a top view schematically showing the optical system of thethree-dimensional shape measuring device 1 shown in FIG. 1.

As shown in FIG. 3, in the three-dimensional shape measuring device 1,as viewed from above, the principal point O2 of the projector 2 and theprincipal point O1 of the camera 3 are spaced apart from each other by aseparation distance vx along the x-axis and a separation distance vyalong the y-axis. Thus, even when two grid patterns in differentdirections from each other are simultaneously projected onto the object9 and images of these grid patterns are picked up, as described above,the Moiré topography optical system shown in FIG. 2 can be formed bothin the x-axis direction and in the y-axis direction. Consequently, thepicked-up images of the two grid patterns are analyzed and the result ofthe analysis can be acquired simultaneously from the two picked-upimages. Thus, even when the object 9 is moving, as described above, theresult of the analysis acquired from the two picked-up images can beused and three-dimensional shape measurement can be performed at highspeed and with high accuracy.

1.2. Measuring Method

The three-dimensional shape measuring method according to the firstembodiment will now be described.

FIG. 4 is a flowchart for explaining the three-dimensional shapemeasuring method according to the first embodiment.

The three-dimensional shape measuring method shown in FIG. 4 includes aprojection step S102, an image pickup step S104, and a computation stepS106.

1.2.1. Projection Step

In the projection step S102, first, a grid pattern is prepared based oneach of lights of two colors included in the three primary colors oflight and is projected by the projector 2. In this specification, thelights of three colors included in the three primary colors of light arereferred to as first light, second light, and third light.

FIG. 5 shows a picked-up image acquired by picking up, by the camera 3,an image of a grid pattern projected by the projector 2, and separatinga first light component. FIG. 6 shows a picked-up image acquired bypicking up, by the camera 3, an image of the grid pattern projected bythe projector 2, and separating a second light component. FIGS. 5 and 6also show arrows representing the x-axis and the y-axis definingdirections of the grid pattern, and dashed lines representing boundariesbetween pixels of the camera 3. A quadrilateral surrounded by dashedlines corresponds to a camera pixel 30.

In FIGS. 5 and 6, the coordinates of the camera pixel 30 are (i, j). Theoptical system is set in such a way that the i-axis of the image pickupelement 31 coincides with the x-axis prescribing a direction of the gridpattern and that the j-axis of the image pickup element 31 coincideswith the y-axis prescribing a direction of the grid pattern.

In FIGS. 5 and 6, the grid pattern of a one-dimensional grid based onthe first light is referred to as “first grid pattern 51” and the gridpattern of a one-dimensional grid based on the second light is referredto as “second grid pattern 52”.

The first grid pattern 51 is a one-dimensional grid extending along they-axis. Specifically, the first grid pattern 51 has a plurality ofstrip-like areas 511 irradiated with the first light with a relativelylow luminance. The strip-like areas 511 extend parallel to the y-axisand are arranged at constant intervals along the x-axis. The first gridpattern 51 also has a plurality of strip-like areas 512 located betweenthe areas 511 and irradiated with the first light with a relatively highluminance. The width of strip-like areas 511 and the width of thestrip-like areas 512 are the same as each other.

The second grid pattern 52 is a one-dimensional grid extending along thex-axis. Specifically, the second grid pattern 52 has a plurality ofstrip-like areas 521 irradiated with the second light with a relativelylow luminance. The strip-like areas 521 extend parallel to the x-axisand are arranged at constant intervals along the y-axis. The second gridpattern 52 also has a plurality of strip-like areas 522 located betweenthe areas 521 and irradiated with the second light with a relativelyhigh luminance. The width of strip-like areas 521 and the width of thestrip-like areas 522 are the same as each other.

The grid pitch of the first grid pattern 51 and the grid pitch of thesecond grid pattern 52 may be different from each other but maypreferably be the same. When the grid pitches are the same, the dynamicranges of measurement based on the two patterns are the same and thismakes it easier to perform measurement and handle the result of themeasurement.

Meanwhile, the number of projector pixels corresponding to one cycle ofthe grid pattern is not particularly limited. That is, the number ofprojector pixels corresponding to the width of the areas 511, 512, 521,522 may be one, or two or more.

In this embodiment, the direction of the first grid pattern 51 and thedirection of the second grid pattern 52 are orthogonal to each other.However, the effect of the embodiment can be achieved when thesedirections intersect each other, if not orthogonal to each other.

In this embodiment, the x-axis of the grid pattern and the i-axis of theimage pickup element 31 correspond to each other and the y-axis of thegrid pattern and the j-axis of the image pickup element 31 correspond toeach other, as described above. However, the direction of arrangement ofprojector pixels of the projector 2 may be inclined to the x-axis andthe y-axis.

FIG. 7 is a schematic view showing the first grid pattern 51 and thesecond grid pattern 52 when the direction of arrangement of projectorpixels 20 is inclined to the x-axis and the y-axis.

In FIG. 7, the direction of arrangement of the projector pixels 20 isinclined to the first grid pattern 51 and the second grid pattern 52. Inother words, the x-axis and the y-axis are inclined to the direction ofarrangement of the projector 2.

In this case, too, the first grid pattern 51 and the second grid pattern52 are prepared and projected in such a way as to be inclined to thedirection of arrangement of the projector pixels 20. This enables thex-axis of the grid pattern and the i-axis of the image pickup element 31to correspond to each other and enables the y-axis of the grid patternand the j-axis of the image pickup element 31 to correspond to eachother, as described above. Thus, the effect of the embodiment can beachieved.

Next, a range corresponding to one cycle of the grid pattern that is atarget of phase analysis will be described.

In this embodiment, the optical system formed by the projector 2 and thecamera 3 is set in such a way that one cycle of the grid pattern appearson N successive pixels of the camera pixels 30. N is an integer equal toor greater than 3.

In the example shown in FIGS. 5 and 6, the first grid pattern 51 has acycle with a length corresponding to eight successive camera pixels 30along the x-axis, and the second grid pattern 52 has a cycle with alength corresponding to eight successive camera pixels 30 along they-axis. Therefore, in the example shown in FIGS. 5 and 6, the opticalsystem is set in such a way that eight camera pixels 30 coincide withone cycle of the first grid pattern 51. Also, in the example shown inFIGS. 5 and 6, the optical system is set in such a way that eight camerapixels 30 coincide with one cycle of the second grid pattern 52. In thisway, in the three-dimensional shape measuring device 1, the opticalsystem is set in such a way that one cycle of the grid pattern projectedby the projector 2 has a length corresponding to an integral multiple ofthe camera pixel 30.

The first light and the second light are lights of at least two colorsof RGB, as descried above. In the projection step S102, these lights aresimultaneously cast and the first grid pattern 51 and the second gridpattern 52 are thus projected simultaneously.

The first light and the second light are lights of two colors of thethree primary colors of light. Therefore, even when these lights arecast as superimposed on each other, the camera 3, which is a three-colorcamera, can separate these lights. This enables high-speed measurementusing lights of two colors.

For such reasons, the projector 2 may preferably be a three-colorseparation projector. The three-color separation projector canseparately emit lights of three colors at all the projector pixels andtherefore has the function of simultaneously projecting the first gridpattern 51 based on the first light and the second grid pattern 52 basedon the second light. Thus, when the camera 3 simultaneously acquires apicked-up image based on the first light and a picked-up image based onthe second light and the control device 4 analyzes the two picked-upimages, separate information can be acquired from the two picked-upimages.

As the three-color separation projector, particularly a three-panelprojector may be preferably used. The three-panel projector has threelight modulation elements corresponding to lights of three colors,respectively. Therefore, the lights of three colors can be separatelymodulated at all the projector pixels and can be cast with high positionaccuracy.

Specifically, as the three-panel projector, for example, atransmission-type 3LCD system, a reflection-type 3LCD system, athree-chip DLP system or the like may be employed. The transmission-type3LCD system uses three transmission-type LCD elements. Thereflection-type 3LCD system uses three reflection-type LCD elements. LCDrefers to liquid crystal display. The three-chip DLP system uses anoptical system that can scan with three lights separately, using threeDMDs. DLP refers to digital light processing. DMD refers to digitalmicromirror device.

In the projector 2, it is desirable to cast lights of three colorsseparately with respect to all the projector pixels, as described above.However, when a pixel group formed of a plurality of projector pixelscan be regarded as one projector pixel, the single light modulationelement 22 may be employed. In this case, measurement can be performedthough the resulting measurement accuracy for three-dimensional shape islower.

1.2.2. Image Pickup Step

In the image pickup step S104, the camera 3 picks up an image of thefirst grid pattern 51 and the second grid pattern 52 projected on theobject 9. A first picked-up image and a second picked-up image acquiredby the image pickup are transmitted from the camera 3 to the computingunit 42.

The camera 3 is a three-color camera having the function of acquiringthe first picked-up image and the second picked-up image separately andsimultaneously, as described above. Therefore, even when the first gridpattern 51 and the second grid pattern 52 are simultaneously projected,image data of these grid patterns that are separated from each other canbe acquired. Thus, based on each of the first picked-up image and thesecond picked-up image, a phase analysis can be performed in the stepdescribed below.

For such reasons, the camera 3 (three-color camera) may preferably havethe three-panel image pickup element 31. The three-panel image pickupelement 31 corresponds to each of the first light, the second light, andthe third light. Therefore, a luminance value can be acquired separatelyfor the first light, the second light, and the third light and with highposition accuracy at all the image pickup pixels.

A specific example of the three-panel image pickup element 31 may be, a3CMOS system, a 3CCD system, a vertical color separation system or thelike. The 3CMOS system uses three CMOS sensors. CMOS refers tocomplementary metal oxide semiconductor. The 3CCD system uses three CCDsensors. CCD refers to charge-coupled device.

The vertical color separation system uses an image pickup element havingthree light receiving layers stacked on each other. A specific exampleis Foveon (trademark registered).

In the camera 3, it is desirable to acquire a luminance value separatelyfor each of lights of three colors at all the camera pixels, asdescribed above. However, when a pixel group formed of a plurality ofcamera pixels can be regarded as one camera pixel, the single imagepickup element 31 may be employed. In this case, measurement can beperformed though the resulting measurement accuracy forthree-dimensional shape is lower.

Meanwhile, in this embodiment, before the foregoing projection stepS102, steps similar to the projection step S102 and the image pickupstep S104 are performed on the reference surface 91 where the object 9is not arranged. In this way, a picked-up image of the reference surface91 is transmitted to the computing unit 42 and the picked-up image orthe result of computation is stored in the storage unit 43.

1.2.3. Computation Step

The computation step S106 further includes a phase analysis step S107, adata selection step S108, and a shape calculation step S109.

1.2.3.1. Phase Analysis Step

In the phase analysis step S107, first, the computing unit 42 performs aphase analysis on the picked-up image. In this embodiment, the computingunit 42 performs the phase analysis, using a known one-pitch phaseanalysis (OPPA) method.

Specifically, first, a luminance value corresponding to one cycle of thegrid pattern is extracted from each of the first picked-up image, whichis a picked-up image of the first grid pattern 51, and the secondpicked-up image, which is a picked-up image of the second grid pattern52.

In FIG. 5, as an example, attention is focused on eight successivepixels along the x-axis including the origin on the x-axis and they-axis. These eight pixels are referred to as “one-pitch grid OP1”. Thisone-pitch grid OP1 is equivalent to the foregoing range corresponding toone cycle with respect to the first grid pattern 51.

In FIG. 6, as an example, attention is focused on eight successivepixels along the y-axis including the origin on the x-axis and they-axis. These eight pixels are referred to as “one-pitch grid OP2”. Thisone-pitch grid OP2 is equivalent to the foregoing range corresponding toone cycle with respect to the second grid pattern 52.

In the one-pitch phase analysis method, a phase analysis is performedsequentially while the set of luminance value data acquired at eachcamera pixel 30 in the one-pitch grid OP1 is shifted by one camera pixeleach time along the x-axis. When all the shifting along the x-axis isfinished, a phase analysis is then performed sequentially while the setof luminance value data acquired at each camera pixel 30 in theone-pitch grid OP1 is shifted by one camera pixel each time along they-axis.

Also, a phase analysis is performed sequentially while the set ofluminance value data acquired at each camera pixel 30 in the one-pitchgrid OP2 is shifted by one camera pixel each time along the y-axis. Whenall the shifting along the y-axis is finished, a phase analysis is thenperformed sequentially while the set of luminance value data acquired ateach camera pixel 30 in the one-pitch grid OP2 is shifted by one camerapixel each time along the x-axis.

The order of these processes is not limited to the above and may bechanged. In this way, phase information can be acquired at all thecamera pixels 30.

First phase information acquired by the phase analysis of the one-pitchgrid OP1 is stored in the storage unit 43 of the control device 4, inthe state of corresponding to the coordinates of one representativecamera pixel in the one-pitch grid OP1. Similarly, second phaseinformation acquired by the phase analysis of the one-pitch grid OP2 isstored in the storage unit 43 of the control device 4, in the state ofcorresponding to the coordinates of one representative camera pixel inthe one-pitch grid OP2.

In this embodiment, the first grid pattern 51 and the second gridpattern 52 orthogonal thereto are both used. Its significance will nowbe described.

FIGS. 8 and 9 show an example of a picked-up image acquired when only agrid pattern in one direction is projected onto four rod-like elementsplaced on a plane. The direction of the rod-like elements differsbetween FIGS. 8 and 9 by approximately 90 degrees. Therefore, in FIG. 8,the direction of the grid pattern and the longitudinal direction of therod-like elements are substantially parallel to each other. On the otherhand, in FIG. 9, the direction of the grid pattern and the longitudinaldirection of the rod-like elements are substantially perpendicular toeach other. As shown in FIGS. 8 and 9, the four rod-like elements havewidths of 1 mm, 2 mm, 3 mm, and 4 mm. The pitch of the grid pattern is1.7 mm.

FIGS. 10 and 11 are graphs prepared by slicing out a part of adistribution of height information found from the picked-up images shownin FIGS. 8 and 9.

From the comparison between FIGS. 10 and 11, it can be seen that, inFIG. 10, the shape of steps representing change in the heightinformation is blunt and therefore the measurement resolution for aposition within the plane of the rod-like elements is low. In FIG. 10,theoretically, the length of one cycle of the first grid pattern 51 isthe measurement resolution.

On the other hand, in FIG. 11, it can be seen that the shape of stepsrepresenting change in the height information is distinct and thereforethe measurement resolution for a position within the plane of therod-like elements is sufficiently high. In FIG. 11, theoretically, thesize of a camera pixel is the measurement resolution.

Based on these observations, it can be understood that the relationshipbetween the shape of the object 9 and the direction of the grid patterninfluences the measurement resolution.

Therefore, in this embodiment, luminance value data is acquired fromboth the first picked-up image and the second picked-up image, at eachcamera pixel. This enables data selection in which one of the picked-upimages is selected for each camera pixel and a phase analysis isperformed thereon. As a result, the three-dimensional shape of theobject surface 92 can be measured with high accuracy regardless of theshape of the object 9.

1.2.3.2. Data Selection Step

In the data selection step S108, the computing unit 42 compares theluminance value data of the first picked-up image with the luminancevalue data of the second picked-up image with respect to each camerapixel. The computing unit 42 then prepares a map for selecting theluminance value data having higher reliability. Based on the preparedmap, the computing unit 42 calculates height information, using theselected picked-up image in a step described below. As such selection ofluminance value data is performed for each camera pixel, a highlyreliable measurement result can be ultimately acquired.

The selection criterion in the above selection is the ability toultimately calculate highly accurate height information, that is, thereliability of the luminance value data. In this embodiment, as anexample of evaluating whether the reliability is good or not, thecorrelativity between a luminance value data set acquired at each camerapixel in the one-pitch grid OP1 and a sine wave generated by calculationwith the phase shifted in the cycle of the first grid pattern 51, andthe correlativity between a luminance value data set acquired at eachcamera pixel in the one-pitch grid OP2 and a sine wave generated bycalculation with the phase shifted in the cycle of the second gridpattern 52, may be employed. The correlativity serves as an indicatorfor evaluating how similar the distribution represented by the luminancevalue data set to the sine wave representing the luminance distributionin the first grid pattern 51.

Therefore, in this step, first, a task of comparing the correlativitybetween the luminance value data set of the one-pitch grid OP1 and thesine wave, with the correlativity between the luminance value set of theone-pitch grid OP2 and the sine wave, is performed. Subsequently, a dataselection map is prepared, based on the result of the comparison.

A more specific procedure for finding the correlativity will now bedescribed.

FIG. 12 explains a procedure for finding the correlativity between aluminance value data set acquired from a one-pitch grid and a pluralityof sine waves generated with the phase shifted.

In FIG. 12, luminance value data of 35 successive camera pixels in they-axis direction is shown as an example. Of these, ten camera pixelsfrom y=0 to y=9 are defined as the one-pitch grid OP1. The luminancevalue data corresponding to the one-pitch grid OP1 is defined as aluminance value data set DS. Therefore, in the example shown in FIG. 12,one cycle of the grid pattern has a length of ten camera pixels.

Meanwhile, in FIG. 12, a sine wave having a cycle corresponding to thelength of ten camera pixels is generated by calculation, and anamplitude value for each camera coordinate is described. The sine waveis expressed by S(y)=sin[2π/10(y+ϕ)], as shown in FIG. 12. In this case,S(y) is the amplitude value and ϕ is the phase of the sine wave.Changing the phase, for example, every 0.5 camera pixels, enablesgeneration of a plurality of sine waves having different phases. In FIG.12, the amplitude values S(y) of a plurality of sine waves generated bychanging the phase every 0.5 camera pixels from ϕ=0 to ϕ=9.5 are shown,as an example. The interval at which the phase is changed is notparticularly limited and may be one camera pixel or less, or 0.5 camerapixels or less.

Next, the correlativities between the luminance value data set DS andthe plurality of sine waves are found and compared. In FIG. 12, of theplurality of sine waves, a range of sine waves to be compared with theluminance value data set DS are defined as “comparison target C”. Tofind the correlativity, the correlation coefficient between theluminance value data set DS and each sine wave included in thecomparison target C is calculated. In this embodiment, the sine wave isused as a reference wave for calculating the correlation coefficient.However, another wave having the same cycle may be used instead of thesine wave.

FIG. 13 is a table showing the absolute values of the correlationcoefficients between the luminance value data set DS for each camerapixel and the plurality of sine waves, calculated from the table shownin FIG. 12, and a maximum value thereof.

The absolute values of the correlation coefficients between theluminance value data set DS for each camera pixel and the plurality ofsine waves, calculated by the foregoing procedure, and the maximum valuethereof, are shown in the row of y=0 in the table shown in FIG. 13.

After numeric values are entered for y=0, the luminance value data setDS and the comparison target C shown in FIG. 12 are then shifted by oneto the positive side along the y-axis. Then, the absolute values of thecorrelation coefficients and the maximum value thereof are calculatedagain and the result of the calculation is shown in the row of y=1 inthe table shown in FIG. 13.

This task is repeated up to the row of y=25 shown in FIG. 13. The tableshown in FIG. 13 is thus prepared. The sine waves with ϕ=5.0 to 9.5shown in FIG. 12 are inverted versions of the sine waves with ϕ=0 to4.5. Based on this, the correlation coefficients between the luminancevalue data set DS and the sine waves with ϕ=5.0 to 9.5 are omitted fromFIG. 13. In the description below, the absolute value of the correlationcoefficient is simply referred to as correlation coefficient.

The maximum value calculated in this way is employed as the correlationcoefficient at each camera pixel.

Similarly, the coefficient correlations and the maximum value thereofare calculated for the one-pitch grid OP2.

By the procedure as described above, the maximum value of thecorrelation coefficients calculated from the one-pitch grid OP1 and themaximum value of the correlation coefficients calculated from theone-pitch grid OP2 can be calculated for each camera pixel.

Therefore, in this step, the maximum value of the correlationcoefficients calculated from the one-pitch grid OP1 and the maximumvalue of the correlation coefficients calculated from the one-pitch gridOP2 are compared with each other for each camera pixel. Then, the highermaximum value is stored in the storage unit 43. That is, in the storageunit 43, a “selected item” representing which of the foregoing firstphase information and the second phase information should be used toacquire ultimate height information is stored in the state of beingassociated with the coordinates of each camera pixel.

When a threshold (reference value) is provided in advance and themaximum value of the correlation coefficients calculated from theone-pitch grid OP1 and the maximum value of the correlation coefficientscalculated from the one-pitch grid OP2 are both lower than thethreshold, ultimate height information may be not outputted for thecorresponding camera pixel. In this case, neither the first phaseinformation nor the second phase information is designated as the“selected item” stored in the storage unit 43. This means that no heightinformation exists for that camera pixel. However, eliminating inadvance the height information having a large margin of error improvesthe usability of three-dimensional shape data and is therefore moreadvantageous than including the height information having a large marginof error.

In FIG. 13, the threshold of the maximum value of the correlationcoefficient is set to 0.95 as an example. The cells where the maximumvalue of the correlation coefficient is less than 0.95 are dotted.Particularly, the cells where the maximum value of the correlationcoefficient is less than 0.90 are dotted relatively densely. As anexample, the information about the camera pixels corresponding to thesecells can be targets of exclusion.

Meanwhile, when the maximum value of the correlation coefficientscalculated from the one-pitch grid OP1 and the maximum value of thecorrelation coefficients calculated from the one-pitch grid OP2 are bothequal to or higher than the threshold, the computing unit 42 stores bothof these maximum values in the storage unit 43. In this case, both thefirst phase information and the second phase information are designatedas the “selected item” stored in the storage unit 43.

In this way, the coordinates of each camera pixel and the information ofthe “selected item” are associated with each other and stored as the“data selection map” in the storage unit 43.

1.2.3.3. Shape Calculation Step

In the shape calculation step S109, the computing unit 42 compares thefirst phase information about the object surface 92 with the first phaseinformation about the reference surface 91 and finds a phase difference.Based on this phase difference, the computing unit 42 calculates firstheight information from the reference surface 91 to the object surface92.

Similarly, the computing unit 42 compares the second phase informationabout the object surface 92 with the second phase information about thereference surface 91 and finds a phase difference. Based on this phasedifference, the computing unit 42 calculates second height informationfrom the reference surface 91 to the object surface 92.

Subsequently, based on the selected item designated in the foregoingdata selection map, one or two of the first height information and thesecond height information are selected for each camera pixel. Ultimateheight information for output is thus acquired. When both the firstheight information and the second height information are selected, forexample, an intermediate value of the two pieces of height informationmay be calculated and this intermediate value may be used as theultimate height information for output.

The timing of applying the data selection map is not limited to theforegoing timing. For example, this timing may be the timing before thefirst phase information and the second phase information are calculated,the timing when the phase difference is calculated, or the timing whenthe height information is calculated.

The three-dimensional shape of the object surface 92 is found in thisway.

The three-dimensional shape measuring method according to thisembodiment has been described. The color of the first light and thecolor of the second light used in this method are suitably selectedaccording to the color of the object 9. For example, before the abovethree-dimensional shape measuring method is performed, thethree-dimensional shape of an object whose shape is known is measured inadvance with the color of light sequentially changed to RGB. Then, thecolor resulting in the highest measurement accuracy may be used as thecolor of light suitable for the object color.

As the first light or the second light, lights of two colors may beused. For example, red light and green light may be used as the firstlight, and blue light may be used as the second light. In this case, afirst picked-up image based on the red light, a first picked-up imagebased on the green light, and a second picked-up image based on the bluelight are acquired. A phase analysis is performed, selecting one of thetwo first picked-up images and the second picked-up image. Thus, thethree-dimensional shape of the object surface 92 can be measured withhigher accuracy. Also, selecting data based on the correlativity betweenthe two first picked-up images can further increase the measurementaccuracy.

The way of allocating RGB to the first light and the second light is notparticularly limited. In an example, a combination in which red light isused as the first light while blue light is used as the second light andin which green light is not used may be employed. Also, a combination inwhich red light is used as the first light and in which blue light andgreen light are used as the second light may be employed.

As described above, the three-dimensional shape measuring methodaccording to this embodiment includes the projection step S102, theimage pickup step S104, and the computation step S106. In the projectionstep S102, the first grid pattern 51 based on the first light and thesecond grid pattern 52 based on the second light are projected onto theobject 9 (target object) in such a way that these grid patternsintersect each other, with the first light and the second light beinglights of two colors included in the three primary colors of light. Inthe image pickup step S104, the camera 3 (three-color camera) picks upan image of the first grid pattern 51 and the second grid pattern 52projected on the object 9 and thus acquires the first picked-up imagebased on the first light and the second picked-up image based on thesecond light. In the image pickup step S106, a phase analysis of thegrid image is performed with respect to at least one of the firstpicked-up image and the second picked-up image, and the heightinformation of the object 9 is thus calculated.

In such a configuration, the phase analysis of each grid image isperformed, using the first grid pattern 51 and the second grid pattern52 intersecting each other. Therefore, the three-dimensional shape ofthe object 9 can be measured at high speed regardless of the shape ofthe object 9. Thus, the three-dimensional shape can be measured withhigh accuracy, for example, even when the object 9 is moving.

In the computation step S106, first, the correlativity between theluminance value in the first picked-up image and the first grid pattern51 and the correlativity between the luminance value in the secondpicked-up image and the second grid pattern 52 are calculated withrespect to the same pixel. Next, the two correlativities, thuscalculated, are compared with each other. Then, a phase analysis isperformed, using the picked-up image having the higher correlativity.

In such a configuration, a phase analysis can be performed on apicked-up image having high reliability. Therefore, thethree-dimensional shape can be measured with higher accuracy.Specifically, for example, selection is made in such a way as not to usefor the phase analysis a picked-up image from which luminance value datacorresponding to one cycle of the grid pattern, which is necessary forthe phase analysis of the grid image, cannot be acquired. Therefore, aresult of measurement from which abnormal height information iseliminated can be acquired. Such a result of measurement is advantageousin that it contains few abnormal values and is therefore easier to use.

As described above, the three-dimensional shape measuring device 1according to this embodiment has the projector 2, the camera 3(three-color camera), and the computing unit 42. The projector 2projects the first grid pattern 51 based on the first light and thesecond grid pattern 52 based on the second light onto the object 9(target object) in such a way that these grid patterns intersect eachother. The camera 3 picks up an image of the first grid pattern 51 andthe second grid pattern 52 projected on the object 9 and thus acquiresthe first picked-up image based on the first light and the secondpicked-up image based on the second light. The computing unit 42performs a phase analysis of the grid image with respect to at least oneof the first picked-up image and the second picked-up image, and thusacquires the height information of the object 9.

Such a configuration enables the three-dimensional shape measuringdevice 1 to perform the phase analysis of each grid image, using thefirst grid pattern 51 and the second grid pattern 52 intersecting eachother, and therefore to measure the three-dimensional shape of theobject 9 at high speed regardless of the shape of the object 9. Thethree-dimensional shape measuring device 1 can also measure thethree-dimensional shape with high accuracy, for example, even when theobject 9 is moving.

2. Second Embodiment

A three-dimensional shape measuring method according to a secondembodiment will now be described.

FIG. 14 is a flowchart for explaining the three-dimensional shapemeasuring method according to the second embodiment. FIG. 15 shows apicked-up image acquired by picking up, by the camera 3, an image of thefirst grid pattern 51, the second grid pattern 52, and an all-pixelirradiation pattern 53 projected by the projector 2, and separating athird light component.

The second embodiment is described below. In the description below, thedifference from the first embodiment is mainly described and thedescription of similar matters is omitted. In FIGS. 14 and 15,components similar to those in the first embodiment are denoted by thesame reference signs.

The second embodiment is similar to the first embodiment, except forusing a third light having a different projection pattern in addition tothe first light and the second light.

The three-dimensional shape measuring method according to thisembodiment includes a projection step S202, an image pickup step S204,and a computation step S206. The computation step S206 includes a phaseanalysis step S207, a data selection step S208, and a shape calculationstep S209.

In the projection step S202, as in the projection step S102 according tothe first embodiment, the first grid pattern 51 and the second gridpattern 52 are projected onto the object 9. In addition to this, in theprojection step S202, the third light is cast in such a way as to coverat least the object 9, and the all-pixel irradiation pattern 53 shown inFIG. 15 is thus projected. At this time, the luminance distribution inthe all-pixel irradiation pattern 53 may preferably be adjusted inadvance in such a way that the luminance value at each camera pixel 30is constant.

The first light, the second light, and the third light are lights of thethree primary colors of light. Even when these lights are cast assuperimposed on each other, the camera 3 can acquire picked-up images ofthese lights that are separated from each other. Therefore, the firstgrid pattern 51, the second grid pattern 52, and the all-pixelirradiation pattern 53 can be simultaneously projected.

Next, in the image pickup step S204, the camera 3 picks up an image ofthe all-pixel irradiation pattern 53 projected on the object 9 inaddition to the first grid pattern 51 and the second grid pattern 52projected on the object 9.

The camera 3 has the function of acquiring a first picked-up image basedon the first light, a second picked-up image based on the second light,and a third picked-up image based on the third light, separately fromeach other.

Next, in the phase analysis step S207 included in the computation stepS206, a phase analysis is performed, as in the phase analysis step S107according to the first embodiment.

Next, in the data selection step S208, a data selection map is prepared,as in the data selection step S108 according to the first embodiment. Inthis embodiment, based on the third picked-up image, information aboutwhether or not to ultimately output height information at each camerapixel 30 is prepared and added to the data selection map.

Specifically, in the data selection map prepared in the firstembodiment, the “selected item” is designated, as described above. Inthe second embodiment, “whether or not to output” to determine whetheror not to output ultimate height information is provided in addition tothe “selected item”. This “whether or not to output” is informationdesignating whether or not to output height information at each camerapixel in the first place. In the data selection step S208, thisinformation is prepared based on the third picked-up image.

The third picked-up image is an image picked up in the state where thethird light is emitted at projector pixels corresponding to all thecamera pixels where the object 9 appears.

Therefore, for example, when the object surface 92 includes an area(shaded area) that is not irradiated with the third light as the thirdlight is blocked by, for example, the object 9 itself, this area isobserved as having an extremely low luminance value. Therefore, if anextremely low luminance value can be detected, a camera pixelcorresponding to the shaded area can be specified based on the extremelylow luminance value.

Also, for example, when the object surface 92 includes an area(reflection area) reflecting the cast third light toward the camera 3,this area is observed as having an extremely high luminance value. Sucha luminance value tends to be observed, for example, when the objectsurface 92 includes a glossy area or the like. When reflection occurs,the luminance value is saturated and a correct luminance value may notbe acquired. Therefore, if an extremely high luminance value can bedetected, a camera pixel corresponding to the reflection area can bespecified based on the extremely high luminance value.

Therefore, in the data selection step S208, whether the luminance valuein the third picked-up image is within a predetermined range or not, isdetermined. When the luminance value is lower than the predeterminedrange or higher than the predetermined range, the information aboutwhether or not to output is added to the data selection map so as not tooutput height information at the corresponding camera pixel, in theshape calculation step S209 described later. This can prevent the outputof the height information having a large margin of error due to theextremely high luminance value or the extremely low luminance value. Asdescribed above, eliminating in advance the height information having alarge margin of error improves the usability of three-dimensional shapedata and is therefore more advantageous than including such heightinformation.

When performing a phase analysis based on the luminance distribution atone grid pitch in one picked-up image, a highly reliable luminance valueneeds to be provided in one cycle of the grid pattern. In view of this,the accuracy of the phase analysis may drop around the camera pixelcorresponding to the shaded area or the reflection area. To cope withthis, in the data selection step S208, the information about whether ornot to output may be added to the data selection map so as to preventthe output of the ultimate height information, not only at the camerapixel corresponding to the shaded area or the reflection area but alsoat peripheral camera pixels pixel in a range corresponding to at leastone cycle of the grid pattern. This can also prevent the output of theheight information at the peripheral camera pixels that are indirectlyaffected by the shaded area or the reflection area.

Next, in the shape calculation step S209, the three-dimensional shape iscalculated, as in the shape calculation step S109 according to the firstembodiment.

The second embodiment, as described above, can achieve effects similarto those of the first embodiment.

The way of allocating RGB to the first light, the second light, and thethird light is not particularly limited in this embodiment, either. Forexample, a combination in which red light is used as the first lightwhile green light is used as the second light and in which blue light isused as the third light may be employed.

In this embodiment, as described above, in the projection step S202, theall-pixel irradiation pattern 53 based on the third light in addition tothe first grid pattern 51 and the second grid pattern 52 is projectedonto the object 9 (target object), with the third light being the lightother than the first light and the second light, of the lights of threecolors included in the three primary colors of light. In the imagepickup step S204, the camera 3 picks up an image of the all-pixelirradiation pattern 53 projected on the object 9 and thus acquires thethird picked-up image. In the computation step S206, when the luminancevalue in the third picked-up image is out of a predetermined range, theheight information of the object 9 is not calculated.

In such a configuration, a camera pixel at which an abnormal luminancevalue occurs can be specified, based on the third picked-up imageacquired by picking up an image of the all-pixel irradiation pattern 53.Thus, a setting can be configured in advance in such a way as not tocalculate height information at this camera pixel. This can prevent theoutput of height information having a large margin of error and thus canprevent a drop in the accuracy of three-dimensional shape data.

The three-dimensional shape measuring method and the three-dimensionalshape measuring device according to the present disclosure have beendescribed, based on the illustrated embodiments. However, thethree-dimensional shape measuring method according to the presentdisclosure is not limited to the embodiments. For example, a step havingany objective may be added to the embodiments. The three-dimensionalshape measuring device according to the present disclosure is notlimited to the embodiments, either. For example, each component in theembodiments may be replaced by a component of any configuration having asimilar function. Alternatively, any component may be added to theembodiments.

What is claimed is:
 1. A three-dimensional shape measuring methodcomprising: projecting a first grid pattern based on a first light and asecond grid pattern based on a second light onto a target object in sucha way that the first grid pattern and the second grid pattern intersecteach other, the first light and the second light being lights of twocolors included in three primary colors of light; picking up, by athree-color camera, an image of the first grid pattern and the secondgrid pattern projected on the target object, and acquiring a firstpicked-up image based on the first light and a second picked-up imagebased on the second light; and performing a phase analysis of a gridimage with respect to at least one of the first picked-up image and thesecond picked-up image and calculating height information of the targetobject.
 2. The three-dimensional shape measuring method according toclaim 1, wherein the first grid pattern and the second grid pattern areprojected by a three-color separation projector.
 3. Thethree-dimensional shape measuring method according to claim 2, whereinthe three-color separation projector is a three-panel projector.
 4. Thethree-dimensional shape measuring method according to claim 1, whereinthe three-color camera has a three-panel image pickup element.
 5. Thethree-dimensional shape measuring method according to claim 1, whereinthe calculating the height information of the target object includes:calculating a correlativity between a luminance value in the firstpicked-up image and the first grid pattern and a correlativity between aluminance value in the second picked-up image and the second gridpattern; comparing the two correlativities that are calculated; andperforming the phase analysis, using the picked-up image having thehigher correlativity.
 6. The three-dimensional shape measuring methodaccording to claim 1, wherein in the projecting the first grid patternand the second grid pattern onto the target object, an all-pixelirradiation pattern based on a third light is projected onto the targetobject, the third light being a light other than the first light and thesecond light, of the lights of three colors included in the threeprimary colors of light, in the acquiring the first picked-up image andthe second picked-up image, an image of the all-pixel irradiationpattern projected on the target object is picked up by the three-colorcamera, and a third picked-up image is acquired, and in the calculatingthe height information of the target object, the height information ofthe target object is not calculated when a luminance value in the thirdpicked-up image is out of a predetermined range.
 7. A three-dimensionalshape measuring device comprising: a projector projecting a first gridpattern based on a first light and a second grid pattern based on asecond light onto a target object in such a way that the first gridpattern and the second grid pattern intersect each other, the firstlight and the second light being lights of two colors included in threeprimary colors of light; a three-color camera picking up an image of thefirst grid pattern and the second grid pattern projected on the targetobject and acquiring a first picked-up image based on the first lightand a second picked-up image based on the second light; and a computingunit performing a phase analysis of a grid image with respect to atleast one of the first picked-up image and the second picked-up imageand calculating height information of the target object.